Search AI Products and News

Recently, the Microsoft Research team, in collaboration with researchers from several universities, released a multimodal AI model called 'Magma'. This model is designed to process and integrate various types of data, including images, text, and videos, to perform complex tasks in both digital and physical environments. As technology continues to advance, multimodal AI agents are being widely applied in fields such as robotics, virtual assistants, and user interface automation. Previous AI systems typically focused on visual-language understanding or robotic operations, making it difficult to combine the two.

Recently, Japan-based startup Rhymes AI unveiled their first artificial intelligence model — Aria. The company claims that Aria is the world's first open-source multimodal mixture of experts (MoE) model. This model not only possesses the capability to handle various input modalities but also claims to be on par with, or even superior to, several well-known commercial models. The design philosophy of Aria aims to provide exceptional understanding and processing capabilities across diverse input forms including text, code, images, and videos.

Recently, Apple's AI research team launched their next-generation family of Multimodal Large Language Models (MLLMs) - MM1.5. This series of models can integrate various data types such as text and images, showcasing new capabilities of AI in understanding complex tasks. Tasks like visual question answering, image generation, and multimodal data interpretation can be better addressed with the help of these models. A major challenge for multimodal models is how to achieve effective interaction between different data types. Previous models often struggled in this aspect.

Recently, the Allen Institute for AI (Ai2) released Molmo, a brand new open source multimodal AI model family that performs exceptionally well, even surpassing OpenAI's GPT-4o, Anthropic's Claude 3.5, and Google's Gemini 1.5 in several third-party benchmark tests. Molmo not only accepts user-uploaded images for analysis but has also been trained using '1000 times less data than its competitors.'

Salesforce has released an open source multimodal AI model named xGen-MM, designed to simultaneously understand and generate various data types such as text and images, significantly changing the way AI research and applications are conducted. The model performs exceptionally well in multiple benchmark tests, demonstrating strong performance compared to similar open source models, and includes pretrained models, datasets, and fine-tuning code. The largest model boasts 4 billion parameters and can handle 'interleaved data,' enabling multitasking, such as answering questions about multiple images at once. The diverse options available in the model reflect the potential of AI.

Meta has stated that it will not launch its upcoming multimodal AI model in the EU due to regulatory concerns, which will prevent European companies from using the model, despite it being released under an open license. Meta spokesperson Kate McLaughlin told The Verge, "In the coming months, we will release the multimodal Camel model, but we will not launch it in the EU due to the unpredictability of the European regulatory environment."The EU has recently set a compliance deadline for the new A

2023 has been one of the most disruptive years in the field of artificial intelligence, with a plethora of generative AI products entering the mainstream. Continuing its transformative journey, generative AI is expected to transition from an exciting topic to real-world applications in 2024. The field of generative AI is rapidly evolving, giving rise to a series of widespread trends that will facilitate the adoption of AI across various industries and its presence in our daily lives. Generative AI models are transcending text creation through the integration of multimodal functionalities.

Unified-IO2 marks a significant breakthrough in the field of artificial intelligence, featuring autoregressive capabilities that can handle various data types including text, images, audio, and video. The innovative single encoder-decoder transformer model overcomes the limitations of previous models in multimodal data processing. It excels in performance, setting new records in GRIT assessments across 35 datasets, particularly surpassing competitors in image generation. Unified-IO2 employs complex and innovative methods, including shared representation spaces and pretrained visual transformers.

["1. Hugging Face has launched a new open source multimodal AI model IDEFIX.","2. IDEFIX is built on the Flamingo visual language model and supports image-text inputs.","3. IDEFIX performs exceptionally well in image-text understanding tasks."]